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Abstract 


The internet is in the middle of a revolution: centralized proprietary services are being replaced with 
decentralized open ones; trusted parties replaced with verifiable computation; brittle location addresses 
replaced with resilient content addresses; inefficient monolithic services replaced with peer-to-peer algo- 
rithmic markets. Bitcoin, Ethereum, and other blockchain networks have proven the utility of decen- 
tralized transaction ledgers. These public ledgers process sophisticated smart contract applications and 
transact crypto-assets worth tens of billions of dollars. These systems are the first instances of internet- 
wide Open Services, where participants form a decentralized network providing useful services for pay, 
with no central management or trusted parties. (PFO has proven the utility of content-addressing by 
decentralizing the web itself, serving billions of files used across a global peer-to-peer network. It lib- 
erates data from silos, 


(permanence to digital information. 


Filecoin is a decentralized storage network that turns cloud storage into an algorithmic market. The 
market runs on a blockchain with agmativesprotocoltoken) (also called “Pileéoim”), @hichimmers eam 
( by providing storage to clients. Conversely, clients spend Filecoin hiring miners to store or distribute 
data. As with Bitcoin, Filecoin miners compete to mine blocks with sizable rewards, but Filecoin mining) 
(power is proportional to active Storage, which directly provides a useful service to clients (unlike Bitcoin 
mining, whose usefulness is limited to maintaining blockchain consensus). This creates a powerful incen- 
tive for miners to amass as much storage as they can, and rent it out to clients. THe protocol weaves) 
(hese amassed resources into a sol healing storage network that anybody in the world can rely on. The 
network achieves robustness by replicating and dispersing content, while automatically detecting and 
(yepairing replica failures. Clients can select replication parameters to protect against different threat 
models. The protocol’s cloud storage network also provides security, as Content iS encrypted end-to-end 
ED while Storagelproviders do nothaveccesstoldecryption|keys. Filecoin works as andneentive 
(layer on top of PFS [I], which can provide storage infrastructure for any data. It is especially useful 
for decentralizing data, building and running distributed applications, and implementing smart contracts. 


This work: 


(a) Introduces the Filecoin Network, gives an overview of the protocol, and walks through several 
components in detail. 

(b) Formalizes decentralized storage network (DSN) schemes and their properties, then constructs File- 
coin as a DSN. 

(c) Introduces a novel class of proof-of-storage schemes called proof-of-replication, which allows proving 
that any replica of data is stored in physically independent storage. 

(d) Introduces a novel useful-work consensus based on sequential proofs-of-replication and storage as a 
measure of power. 

(e) Formalizes verifiable markets and constructs two markets, a Storage Market and a Retrieval Market, 
which govern how data is written to and read from Filecoin, respectively. 

(£) Discusses use cases, connections to other systems, and how to use the protocol. 


Note: Filecoin is a work in progress. Active research is under way, and new versions of this paper will 
appear at https://filecoin.io. For comments and suggestions, contact us at research@filecoin.io. 
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1 Introduction 


Filecoin is a protocol token whose blockchain runs on a novel proof, called Proof-of-Spacetime, where blocks ) 
are created by miners that are storing data. Filecoin protocol provides a data storage and retrieval service 

via a network of independent storage providers that does not rely on a single coordinator, where: (1) clients 
` pay to store and retrieve data, (2) Storage Miners earn tokens by offering storage (3) Retrieval Miners earn 
tokens by serving data. 


1.1 Elementary Components 


The Filecoin protocol builds upon four novel components. 


1. Decentralized" Storage Network (DSN): We provide an abstraction for network of independent 


storage providers to offer storage and retrieval services (in Section [2). Later, we present the Filecoin 
protocol as an incentivized, auditable and verifiable DSN construction (in Section [4). 


2. Novel Proofs*of-Storage? We present two novel Proofs-of-Storage (in Section B}: (1) Proof-of- 
Replication allows storage providers to prove that data has been replicated to its own uniquely dedicated 
physical storage. Enforcing unique physical copies enables a verifier to check that a prover is not 
deduplicating multiple copies of the data into the same storage space; (2) Proof-of-Spacetime allows 
storage providers to prove they have stored some data throughout a specified amount of time. 


3.( Verifiable Markets: We model storage requests and retrieval requests as orders in two decentralized 
verifiable markets operated by the Filecoin network (in Section [5). Verifiable markets ensure that 
payments are performed when a service has been correctly provided. We present the Storage Market 
and the Retrieval Market where miners and clients can respectively submit storage and retrieval orders. 


4. Useful Proof- of Work: We show how to construct a useful Proof-of-Work based on Proof-of- 
Spacetime that can be used in consensus protocols. Miners do not need to spend wasteful computation 
to mine blocks, but instead must store data in the network. 


1.2 Protocol Overview 


e The Filecoin protocol is a Decentralized Storage Network construction built on a blockchain and with 
a native token. Clients spend tokens for storing and retrieving data and miners earn tokens by storing 
and serving data. 


e The Filecoin DSN handle storage and retrieval requests respectively via two verifiable markets: the 
Storage Market and the Retrieval Market. Clients and miners set the prices for the services requested 
and offered and submit their orders to the markets. 


e The markets are operated by the Filecoin network which employs Proof-of-Spacetime and Proof-of- 
Replication to guarantee that miners have correctly stored the data they committed to store. 


e Finally, miners can participate in the creations of new blocks for the underlining blockchain. The 
influence of a miner over the next block is proportional to the amount of their storage currently in use 
in the network. 


A sketch of the Filecoin protocol, using nomenclature defined later within the paper, is shown in Figure 
accompanied with an illustration in Figure [2] 


1.3 Paper organization 


The remainder of this paper is organized as follows. We present our definition of and requirements for a 
theoretical DSNscheme in Section 2. In Section 3 we motivate, define, and present our Proof-of-Replication 
and Proof-of-Spacetime protocols, used within Filecoin to cryptographically verify that data is continuously 
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stored in accordance with deals made. Section 4 describes the concrete instantiation of the Filecoin DSN, 
describing data structures, protocols, and the interactions between participants. Section 5 defines and de- 
scribes the concept of Verifiable Markets, as well as their implementations, the Storage Market and Retrieval 
Market. Section 6 motivates and describes the use of the Proof-of-Spacetime protocol for demonstrating and 
evaluating a miner’s contribution to the network, which is necessary to extend the blockchain and assign 
the block reward. Section 7 provides a brief description of Smart Contracts within the Filecoin We conclude 
with a discussion of future work in Section 8. 


Filecoin Protocol Sketch 


Network 
at each epoch t in the ledger £: 


1. for each new block: 
check if the block is in the valid format 
check if all transactions are valid 
check if all orders are valid 
check if all proofs are valid 
check if all pledges are valid 
discard block, if any of the above fails 
2. for each new order O introduced in t 
(a) add O to the Storage Market’s orderbook. 
(b) if O is a bid: lock O.funds 
(c) if O is an ask: lock O.space 
(d) if O is a deal: run Put.AssignOrders 
3. for each O in the Storage Market’s orderbook: 
(a) check if O has expired (or canceled): 
e remove O from the orderbook 
e return unspent O.funds 
e free O.space from AllocTable 
(b) if O is a deal, check if the expected proofs 
exist by running Manage.RepairOrders: 
e if one missing, penalize the M’s pledge 
collateral 
e if proofs are missing for more than Afaut 
epochs, cancel order and re-introduce it 
to the market 
e if the piece cannot be retrieved and re- 
constructed from the network, cancel or- 
der and re-fund the client 
Client 
at any time: 
1. submit new storage orders via Put.AddOrders 
(a) find matching orders via Put.MatchOrders 
(b) send file to the matched miner M 
2. submit new retrieval orders via Get.AddOrders 
(a) find matching orders via Get.MatchOrders 
(b) create a payment channel with M 
on receiving Odeal from Storage Miners M 


1. sign O deal 


2. submit the signed Odea to the blockchain via 


Put.AddOrders 
on receiving (p;) from Retrieval Miners M: 


1. verify that (p;) is valid and it was requested 
2. send a micropayment to M 


Storage Mine 


at any time: 


1. renew expired pledges via Manage.PledgeSector 
2. pledge new storage via Manage.PledgeSector 
3. submit a new ask order via Put.AddOrder 


at each epoch t: 


1. for each Oask in the orderbook: 
(a) find matched orders via Put.MatchOrders 
(b) start a new deal by contacting the matching 

client 

2. for each sector pledged: 
(a) generate proof of via 
Manage.ProveSector 

(b) if time to post the proof (every Aproot 
epochs), submit it to the blockchain 


storage 


on receiving piece p from client C: 
1. check if the piece is of the size specified in the 
order Obiad 
2. create Odeai and sign it and send it to C 
3. store the piece in a sector 
4. if the sector is full, run Manage.SealSector 


Retrieval Mine 


at any time: 

1. gossip ask orders to the network 

2. listen to bid orders from the network 
on retrieval request from C: 

1. start payment channel with C 

2. split data in multiple parts 

3. only send parts if payments are received 


Figure 1: Sketch of the Filecoin Protocol. 
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Figure 2: Illustration of the Filecoin Protocol, showing an overview of the Client-Miner interactions. The 
Storage and Retrieval Markets shown above and below the blockchain, respectively, with time advancing from 
the Order Matching phase on the left to the Settlement phase on the right. Note that before micropayments 
can be made for retrieval, the client must lock the funds for the microtransaction. 


2 Definition of a Decentralized Storage Network 


We introduce the notion of a Decentralized Storage Network (DSN) scheme. DSNs aggregate Storage offered: 


Gig. Coordination isdecentralized and desnot require trusted parties: the secure operation of theses 


systems is achieved through protocols that coordinate and verify operations carried out by individual parties. 
DSNs can employ different strategies for coordination, including Byzantine Agreement, gossip protocols, or 
CRDTs, depending on the requirements of the system. Later, in Section [4] we provide a construction for 
the Filecoin DSN. 


Definition 2.1. A DSN scheme M is a‘tuple of protocols run by storage providers and clients: 


eo (Put(data) S key: Clients execute the Put protocollt6 store datalimderaluniquelidentifierskey» 
e Geike =at: Clients execute the Get protocol €6lretrieve data that is currently stored using key? 
e Manage()? The network of participants§@oordinates) via the Manage protocol to: Control the available) 


. The Manage protocol is run 
by storage providers often in conjunction with clients or a network of auditor4"] 


A DSN scheme IT must guarantee data integrity and retrievability as well as tolerate management and storage 
(faults defined in the following sections. 


2.1 Fault tolerance 


2.1.1 Management faults 
We define management faults to be byzantine faults Caused by participants in the Manage protocol) A DSN 


scheme relies on the fault tolerance of its underlining Manage protocol. Violations on the faults tolerance 
assumptions for management faults Can compromise liveness and safety of the system. 

For example, consider a DSN scheme II, where the Manage protocol requires Byzantine Agreement (BA) 
to audit storage providers. In such protocol, the network receives proofs of storage from storage providers 
and runs BA to agree on the validity of these proofs. If the BA tolerates up to f faults out of n total 
nodes, then our DSN can tolerate f < n/2 faulty nodes. On violations of these assumptions, audits can be 
compromised. 


2.1.2 Storage faults 


We define storage faults to be byzantine aults that prevent clients from retrieving the data? i.e. Storage 


Miners lose their pieces, Retrieval Miners stop serving pieces. A successful Put execution is (f,m)-tolerant 
if it results in its input data being stored in m independent storage providers (out of n total) and it can 
tolerate up to f byzantine providers. The parameters f and m depend on protocol implementation; protocol 
designers can fix f and m or leave the choice to the user, extending Put(data) into Put(data, f, m). AvGet) 


For example, consider a simple scheme, where the Put protocol is designed such that each storage provider 
stores all of the data. In this scheme m = n and f =m — 1. Is it always f = m — 1? No, some schemes can 
(be designed using erasure coding) where each storage providers store a special portion of the data, such that 


x out of m storage providers are required to retrieve the data; in this case f = m — zx. 


2.2 Properties 


We describe the two required properties for a DSN scheme and then present additional properties required 
by the Filecoin DSN. 


lIn the case where the Manage protocol relies on a blockchain, we consider the miners as auditors, since they verify and 
coordinate storage providers 


2.2.1 Data Integrity 


This property requires that no bounded adversary A can convince clients to accept altered or falsified data 
at the end of a Get execution. 


Definition 2.2. A DSN scheme II provides @atavintegrity it for any successful) Put execution for some data 


2.2.2 Retrievability 


This property captures the requirement that, Given Ourlfault@tolerancelassumptionspof II, if some data has 
been successfully stored in II and storage providers continue to follow the protocol, then clients can eventually 
retrieve the data. 


Definition 2.3. A DSN scheme II provides qetmievabilitysifsforyanyysuccesstul)Putiexecution|foridatalunder> 
key, there exists a successful Get execution for key for which a client retrieves dataf 


2.2.3 Other Properties 


DSNs can provide other properties specific to their application. We define three key properties required by 
the Filecoin DSN: public verifiability, auditability, and incentive-compatibility. 


Definition 2.4. A DSN scheme II is pubiehpwerifiableiiMtor cad Successhil) Put) the metwork or storage 
providers can generate a proof that the data is currently being stored. The Proof-of-Storage must convince | 


Definition 2.5. A DSN scheme II is@uditableif it generates \awerifiable trace) of operation that) can\be) 
checked in the future to confirm storage was indeed stored for the right duration of time. 


Definition 2.6. A DSN scheme II is@neentive=compatiblé, if storage providers are rewarded for Successiully) 
Offering Storage and retrieval Service) or penalized for misbehaving; such that the storage providers’ dominant 


strategy is to store data. 


?This definition does not guarantee every Get to succeed: if every Get eventually returns data, then the scheme is fair. 
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3 Proof-of-Replication and Proof-of-Spacetime 


In the Filecoin protocol, storage providers must convince their clients that they stored the data they were 
paid to store; in practice, storage providers will generate Proofs-of-Storage (PoS) that the blockchain network 
(or the clients themselves) verifies. 


In this section we motivate, present and outline implementations for the Proof-of-Replication (PoRep) and 
Proof-of-Spacetime (PoSt) schemes used in Filecoin. 


3.1 Motivation 


‘Proofs-of-Storage (PoS) schemes such as Provable Data Possession (PDP) [2] and Proof-of-Retrievability 
(PoR) [3 [4] schemes@ll6w ase? (i.e. the verifierV) whojeutsourcesdatalD toalserver (i.e. the prover P) t 
@epeatedly check if the server is Still) Storine"D. The user can verifytheuntesrity of the data outsourced to a 
server in a very efficient way @moreélemciently than downloading the data) The server generates probabilistic» 
r ni li i iian i . 1 iati 
q@gchallenge/response(protoco) with the user. 


PDP and PoR schemes only guarantee that a prover had possession of some data at the time of the chal- 
lenge/response. In Filecoin, wemeed stronger guarantees’ to prevent three types of attacks that malicious 
miners could exploit to get rewarded for storage they are not providing: Sybil attack, outsourcing attacks, 
generation attacks. 


© SybiltAlti@@hS Malicious miners could prevenditoystores (andigetipaidfor)mmiorereopiesithanntherones 


e OutsourcingAttacks: Malicious miners could commit to store more data than the amount they can 


© Generation Attacks: Malicious miners could Gaim to be storing a large amount of datalwhich they 


If the program is smaller than 
the purportedly stored data, this inflates the malicious miner’s likelihood of winning a block reward in 
Filecoin, which is proportional to the miner’s storage currently in use. 


3.2 Proof-of-Replication 


Proof-of-Replication (PoRep) is a novel Proof-of-Storage which allows a server (i.e. the prover P) to convince 
a user (i.e. the verifier V) that some data D has been replicated to its own uniquely dedicated physical 
storage. Our scheme is an interactive protocol, where the prover P: (a) commits to store n distinct replicas 
(physically independent copies) of some data D, and then (b) convinces the verifier V, that P is indeed 
storing each of the replicas via a challenge/response protocol. To the best of our knowledge, PoRep improves 
on PoR and PDP schemes, preventing Sybil Attacks, Outsourcing Attacks, and Generation Attacks. 

This doesn't say anything about downloading the data. 
Note. For a formal definition, a description of its properties, and an in-depth study of Proof-of-Replication, 
we refer the reader to [5]. 


Definition 3.1. (PPO0LOHREplication) A PoRep scheme @fablésiamlemicicnt prover P to convince a verifier 
V that P is storing a replica R, a physical independent copy of some data D, unique to P. A PoRep protocol 
is characterized by a tuple of polynomial-time algorithms: 


Chi v OD 

PoRep.Setup is used to generate a replica R, and give P and V the necessary 

aommation to run PoRep Prove and PoRep Verify) Some schemes may require the prover or interaction 
with a third party to compute PoRep.Setup. 
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e PoRep.Prove(Sp,R,c) => m°, where c is a random challenge issued by a verifier V, and 7° is a proof- 
(thiataiprover has access to" Ral Specihielreplica@of"D» PoRep.Prove is run by P to produce a 7° for V. 


o POREDNE (SPE EOI}, which checkshwhetherlalproonisiworreet. PoRep. Verify is run by V and 


convinces V whether P has been storing R. 


3.3 Proof-of-Spacetime 


Proof-of-Storage schemes allow a user to check if a storage provider is storing the outsourced data at the time 
of the challenge. How can we use PoS schemes to prove that some data was being stored throughout a period 
of time? A natural answer to this question is to require the user to repeatedly (e.g. every minute) send 
challenges to the storage provider. However, the communication complexity required in each interaction can 
be the bottleneck in systems such as Filecoin, where storage providers are required to submit their proofs to 
the blockchain network. 


To address this question, we introduce a new proof, Proof-of-Spacetime, where a verifier can check if a prover 
is storing her/his outsourced data for a range of time. The intuition is to require the prover to (1) generate 


SequentialiRroofstof2Stonage (in our case Proof-of Replication), @syemwaystoydeterminestimes(2)srecursivel> 
compose the executions to generate a short proof. 


Definition 3.2. (Proof-of-Spacetime) A PoSt scheme enables an éfficient prover P to convince a verifier 
oa pP is storing some data D for sometime’. A PoSt is characterized by a tuple of polynomial-time 


algorithms: 


e sco where Sp and Sy are scheme-specific setup variables for P and V, A is a 
. PoSt.Setup is used to give "Dland ithe necessary information| uP SEPOV 


a SEE Some schemes may require the prover or interaction with a third party to compute 


PoSt.Setup. 


© PoSt.Prove(Sp,D,c,t) => n°, where c is a random challenge issued by a verifier V, and 7° is a proof 
that a prover has access to Dror Some time M PoSt.Prove is run by P to produce a 7° for V. 


© POSE Very Ope Cet P which Aeee. PoSt.Verify is run by V and 


convinces VY whether P has been storing D for some time t. 


3.4 Practical PoRep and PoSt 


We are interested in practical PoRep and PoSt constructions that can be deployed in existing systems and do 
not rely on trusted parties or hardware. We give a construction for PoRep (see Seal-based Proof-of-Replication 
in [5}) that requires a very slow sequential computation Seal to be performed during Setup to generate a 
replica. The protocol sketches for PoRep and PoSt are presented in Figure 4| and the underlying mechanism 
of the proving step in PoSt is illustrated in Figure [8] 


3.4.1 Cryptographic building blocks” 


(Collision=réesistantthashing) We use a collision resistant hash function CRAS FOS FOO, We 
also use a collision resistant hash function MerkléCRA, which divides a string im multiple parts) Construct a) 
binary tree and recursively apply CRH and outputs the root. 


@ESNARKsS) Our practical implementations of PoRep and PoSt rely on zero-knowledge Succinct Non- 
interactive ARguments of Knowledge (zk-SNARKs) [6] [7] 8]. Because zk-SNARKs are 8uéeinet, procis aro 
(very Short and easy to verify. More formally, let L be an NP language and C be a decision circuit for L. 
A trusted party conducts a one-time setup phase that results in two public keys: a proving key pk and a 
verification key vk. The proving key pk enables any (untrusted) prover to generate a proof 7 attesting that 
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x € L for an instance x of her choice. The non-interactive proof m is both zero-knowledge and proof-of- 
knowledge. Anyone can use the verification key vk to verify the proof 7; in particular zk-SNARK proofs are 
publicly verifiable: anyone can verify 7, without interacting with the prover that generated m. The proof 7 
has constant size and can be verified in time that is linear in |z]. 


(AzZeSNARK for circuit satisfiability is a triple of polynomial-time algorithms 


e KeyGen(1*,C) — (pk, vk). On input security parameter \ and a circuit C, KeyGen probabilistically 
Samples pk and Vk. Both keys areq@ubliSHé@ as public parameters and can be used 6lprove/ Verity 
membership in Lc. 


° Prove(pk, x, w) > 7. On input pk and input x and Witness for thé NP Statement w, the prover Prove 
outputs a non-interactive proof m for the statement x € Lo. 


e (Werify(Vk, an) s {0I}. On input vk, an input z, and a proof m, the verifier Verify outputsaMif 
D 


We refer the interested reader to [6} for formal presentation and implementation of zk-SNARK systems. 
Generally these systems require the KeyGen operation to be run by a trusted party; novel work on Scalable 
Computational Integrity and Privacy (SCIP) systems [9] shows a promising direction to avoid this initial 
step, hence the above trust assumption. 


3.4.2 Seal operation 


The role of the Seal operation is to Mforce replicas to be physically independent copies by requiring provers 
to store a pseudo-random permutation of D unique to their public key, such that committing to store n- 
replicas results im! dedicating disk space for mM independent replicas (hence n times the storage size of a 

replica) and (2) to force the generation of the replica during PoRep.Setup to take substantially longer than 
(@he time expected for responding to a challenge For a more formal definition of the Seal operation see [5]. 


The above operation can be realized with = 
(thelhonestichallenge-prove-verify sequence. Note that it 


is distinguishably more expensive than running Prove with random access to R. 
3.4.3 Practical PoRep construction 


This section describes the construction of the PoRep protocol and includes a simplified protocol sketch in 
Figure [4] implementation and optimization details are omitted. 


(Creating a Répliéa. The Setup algorithm generates a replica Via thé Séa) operation and a proof that it was 
correctly generated. The prover generates the replica and sends the outputs (excluding R) to the verifier. 


Setup 

e INPUTS: 
— prover key pair (pkp, skp) 
— prover SEAL key pkseaL 
— data D 


e OUTPUTS: replica R, Merkle root rt of R, proof tseaL 


Proving Storage. The Prove algorithm generates a proof of storage for the replica. The prover f@céives a) 
@andomchallengeye; from the verifier, 
` rt; the prover generates a proof of knowledge about R, and its Merkle path leading up to rt. 
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Prove 

© INPUTS: 
— prover Proof-of-Storage key pkpos 
— replica R 
— random challenge c 


e OUTPUTS: a proof Tpos 


‘Verifying the Proofs. The Verify algorithm checks the validity of the proofs of storage given the Merkle 
root of the replica and the hash of the original data. Proofs are publicly verifiable: nodes in the distributed > 


Verify 
e INPUTS: 
— prover public key, pkp 
verifier SEAL and POS keys vkseaL, vkpos 
— hash of data D, hp 
— Merkle root of replica R, rt 


— random challenge, c 
— tuple of proofs, (7sEaL, Tpos) 


e OUTPUTS: bit b, equals 1 if proofs are valid 


3.4.4 Practical PoSt construction 


This section describes the construction of the PoSt protocol and includes a simplified protocol sketch in Fig- 
ure |4| implementation and optimization details are omitted. The Setup and Verify algorithm are equivalent 
to the PoRep construction, hence we describe here only Prove. 


“Proving space and time. The Prove algorithm generates a Proof-of-Spacetime for the replica. The prover 
receives a random challenge from the verifier and generate Proofs-of-Replication in sequence, using the output 
of a proof as an input of the other for a specified amount of iterations t (see Figure/3). 


Prove 
e INPUTS: 


— prover PoSt key pkpgst 


replica R 
— random challenge c 
— time parameter t 


e OUTPUTS: a proof Tpost 
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Figure 3: Illustration of the underlying mechanism of PoSt.Prove showing the iterative proof to demonstrate 
storage over time. 


© 
3.5 Usage in Filecoin 


The Filecoin protocol employs Proof-of-Spacetime to audit the storage offered by miners. To use PoSt in 
Filecoin, weGddify our Scheme to De noninteractive since there is no designated verifier, and qelwantlaiy 
MeMmberorthenetworkito belabletelverify Since our verifier runs in the public-coin model, We can extrach 
randomness from the blockchain to issue challenges. 


randomness from the blockchain? 
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Filecoin PoRep protocol 


Setup 
e INPUTS: 
— prover key pair (pkp, skp) 
— prover SEAL key pkseaL 
— data D 
e OUTPUTS: replica R, Merkle root rt of R, proof 


TSEAL 


Compute hp := CRH(D) 

Compute R := Seal’ (D, skp) 

Compute rt := MerkleCRH(R) 

Set  := (pkp, hp, rt) 

Set w := (skp, D) 

Compute sear := SCIP.Prove(pksea Z, W ) 


Output R, rt, TSEAL 


s Oa A UOU Ne 


Prove 

e INPUTS: 
— prover Proof-of-Storage key pkpos 
— replica R 
— random challenge c 

e OUTPUTS: a proof Tpos 


1) Compute rt := MerkleCRH(?) 

2) Compute path := Merkle path from rt to leaf R, 
3) Set Z := (rt,c) 

4) Set w := (path, Re) 

5) Compute mpos := SCIP.Prove(pkpos, Z, W ) 

6) Output zpos 

Verify 

e INPUTS: 


— prover public key, pkp 
— verifier SEAL and POS keys vkseat, vkpos 
— hash of data D, hp 
— Merkle root of replica R, rt 
— random challenge, c 
— tuple of proofs, (7seaL, mpos) 
e OUTPUTS: bit b, equals 1 if proofs are valid 
) Set și := (pkp, ho, rt) 
) Compute bı := SCIP.Verify(vkseaL, 21 , 7SEAL) 
3) Set #3 := (rt, c) 
) Compute bz := SCIP.Verify(vkpos, £2 , mpos) 
) Output bı A be 


Filecoin PoSt protocol 


Setup 

e INPUTS: 
— prover key pair (pkp, skp) 
— prover POST key pair pkpost 
— some data D 

e OUTPUTS: replica R, Merkle root rt of R, proof 
TSEAL 

1) Compute R, rt, TmseaL := 
skp, PkseqL, D) 

2) Output R, rt, TSEAL 


PoRep.Setup(pkp, 


Prove 
e INPUTS: 
— prover PoSt key pkpost 
— replica R 
— random challenge c 
— time parameter t 
e OUTPUTS: a proof mpost 
) Set mpost := L 
2) Compute rt := MerkleCRH(R) 
) For i = 0...t: 
a) Set c := CRH(post||el|2) 
b) Compute zpos := PoRep.Prove(pkpos; R, c’) 
c) Set Z := (rt, c, i) 
d) Set w := (Tpos, TposT) 
e) Compute Tpost := SCIP.Prove(pkpos7, Z, w ) 


4) Output Tpost 


Verify 
e INPUTS: 
— prover public key pkp 
— verifier SEAL and POST keys vkseaL, vkpost 
— hash of some data hp 
— Merkle root of some replica rt 
— random challenge c 
— time parameter t 
— tuple of proofs (aseaL, Tpost) 
e OUTPUTS: bit b, equals 1 if proofs are valid 
) Set #1 := (pkp, ho, rt) 
) Compute bı := SCIP.Verify(vkseaL, £i , 7SEAL) 
3) Set £2 := (rt, c, t) 
) Compute bz := SCIP.Verify(vkpost, £2 , TPosT) 
) Output bı A be 


Figure 4: Proof-of-Replication and Proof-of-Spacetime protocol sketches. Here CRH denotes a collision- 


resistant hash, # is the NP-statement to be proven, and w is the witness. 


15 


4 Filecoin: a DSN Construction 


The Filecoin DSN is a decentralized storage network that is auditable, publicly verifiable and designed on 
incentives. Clients pay a network of miners for data storage and retrieval; miners offer disk space and 
bandwidth in exchange of payments. Miners receive their payments only if the network can audit that their 
service was correctly provided. 


In this section, we present the Filecoin DSN construction, based on the DSN definition and Proof-of- 
Spacetime. 


4.1 Setting 
4.1.1 Participants 


Any user can participate as a Client, a Storage Miner, and/or a Retrieval Miner. 


e (Storage Miners provide data Storage to the network. Storage Miners participate in Filecoin by ferme 
¢heirdisk\spacéland serving Put requests: To become Storage Miners, Wers must pledge thein Storage 
by depositing collateral proportional to it. Storage Miners respond to Put requests by committing to 
store the client’s data for a specified time. Storage Miners generate Proofs-of-Spacetime and submit 
them to the blockchain to prove to the Network that they are storing the data through time. In case 
of invalidyoramissingyproofs, Storage Miners are penalized (anidiloose partOltheimcollakeraly Storage 
Miners are also eligible to mine new blocks, and in doing so they hence receive the mining reward for 
creating a block andstransactiomfees for the transactions included in the block. 


e Retrieval Miners provide data retrieval to the Network. Retrieval Miners participate in Filecoin by 
CHING MdAtamthatusersimequestiiviayGet. Unlike Storage Miners, they are m@tinequitedltolplédge) 
cOmimititolstoreldataylor/providelprooofStorage. It is natal for Storage Minersifolalselparticipate 


as Retrieval Miners. Retrieval Miners can obtain pieces directly from clients, or from the Retrieval 
“Market. 


4.1.2 Th NEMO e N 


We personify all the users that rum Filecoin full nodes as one single abstract entity: The Network. The 
Network acts as an Gntermmediary|ithatirims the! Manage protocol; informally, atlevery new block inthe 
Filecoin blockchain, full nodes manage the available storage, validate pledges, audit the storage proofs, and 
repair possible faults. 


4.1.3 The Dedger 


Ourprotocolisapplied on top of a ledgerbased currency; for generality we refer to this as the Ledger, @. At 
any given time t (referred to as epoch), all users have access to £+, the ledger at epoch t, which is a sequence 
of transactions. The Filecoin DSN protocol can be implemented on any ledger 
(that allows for the verification of Filecoin’s proofs;)we show how we can construct a ledger based on useful 


work in Section [6] 


4.1.4 The Markets? 


Demand and supply of storage meet at the two Filecoin Markets: Storage Market and Retrieval Market. 
The markets are two decentralized exchanges and are explained in detail in Section [5] In brief, Clients ana 
‘miners set the prices for the services they request or provide by submitting orders to the respective markets. 
The exchanges provide a way for clients and miners to see matching offers and initiate deals. By running 
the Manage protocol, the Network guarantees that miners are rewarded and clients are charged if the service- 
requested has been successfully provided. | 
3¢ < t' implies that Ly is a prefix of £L, 
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4.2 Data Structures 
Pieces. A piece is some /part/of data) that/a client is|storing inthe DSN» For example, data can be deliber- 


ately divided into many pieces and each piece can be stored by a different set of Storage Miners. 


GS8EtGES? A sector is some disk spaco that a Storage Miner provides to the network Miners store pieces 
from clients in their sectors and earn tokens for their services. In order to store pieces, Storage Miners must 
pledge their sectors to the network. 


ANSCAtiONTABIA) The AlléeTAbl is a data structure that Keepsitrackioflpicces land ithicin assignedSectorsy 
The AllocTable is updated at every block in the ledger and its Merkle root is stored in the latest block. In 
practice, the table is used to keep the state of the DSN, allowing for quick look-ups during proof verification. 


For more details, see Figure [5] wouldn't this table be huge? 


Orders. An order is a statement of intent to request or offer a service. 
markets to request a Service)(resp. Storage Market for storing data and Retrieval Market for retrieving data) 
Gia Miners submit ask orders to Offer a Service® The order data structures are shown in Figure[10| The 
Market Protocols are detailed in Section [5] 


@Fderbosk) Orderbooks areSetSGfGLAG See the Storage Market orderbook in Section and Retrieval 
Market orderbook in Section for details. 


Pledge) A pledge is@ commitment to Offer storage (specifically @yseetor) to the network. Storage Miners 
must submit their pledge to the ledger in order to start accepting orders in the Storage Market. A pledge 


consists of the size of the pledged sector and the collateral deposited by the Storage Miner (see Figure[5] for 


more details). 


Data Structures 


Pledge Allocation 

pledge := (size, coll) m, allocTable: {Mj  — (allocEntry..allocEntry), M2.. } 
e size, the size of the sector being pledged. E 
e coll, the collateral specific to this pledge that allocEntry: (sid, orders, last, missing) 
Mi; deposits. sid, sector id 

Ot, currently valid deal, ask, bid orders. 
Orderbook 
OrderBook: (0'..0”) 


e ©’, currently valid deal, ask, bid orders. 


orders, set of orders {Ojea -C deai} 
last, last proof of storage in the ledger £ 


missing, counter for missing proofs 


Figure 5: Data Structures in a DSN scheme 


4.3 Protocol 


In this Section, we give an overview of the Filecoin DSN by describing the operations performed by the 
clients, the Network and the miners. We present the methods of the Get and the Put protocol in Figure [m] 
and the Manage protocol in Figure [8} An example protocol execution is shown in Figure a The overall 
Filecoin Protocol is presented in Figure [I] 


4.3.1 Client Cycle 


We give a brief overview of the client cycle; an in-depth explanation of the following protocols is given in 
Section [5] 
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1. Put: Client stores data in Filecoin. 


Clients can store their data by paying Storage Miners in Filecoin tokens. The Put protocol is described 
in detail in Section 


< sends the piece to the miner. Both parties sign a deal order and submit it to the Storage Market 
orderbook. 


multiplelordérs\(6r Specifying alreplication factor in-the order). Higher redundancy results in a higher 


tolerance of storage faults. 
2. Get: Client retrieves data from Filecoin. 


Clients can retrieve any data stored in the DSN (by paying) Retrieval Miners in Filecoin tokens: The 
Get protocol is described in detail in Section [5.3} 


A client initiates the Get protocol by submitting a bid order to the Retrieval Market orderbook (by 
gossiping their order to the network). WH@ilalimatching!ash\order fromlminerstisifoundyitherclient» 
GSES Pia OM THEMEN When received, both parties sien a deal order and submit i €0 thé 


4.3.2 Mining Cycle (for Storage Miners) 


We give an informal overview of the mining cycle. 
1. Pledge? Storage Miners pledge to provide storage to the Network. 


Storage Miners pledge their storage to the network Byldépositing Collateral via a pledge transaction in» 
the blockchain, via Manage.PledgeSector. The collateral is deposited for the time intended to provide 
the seo, an tea Ie net generates oo ova the data hey sno 


C in the Storage 
Market: they set their price and add an ask order to the market’s orderbook. 


Manage.PledgeSector 
e INPUTS: 


— current allocation table allocTable 
— pledge request pledge 
e OUTPUTS: allocTable’ 


2. Receive Orders: Storage Miners get storage requests from the Storage Market. 


Once the pledge transaction appears in the blockchain (hence in the AllocTable), miners can offer their 
storage in the Storage Market: they 


Put.AddOrders 
e INPUTS: list of orders 0!..0” 


e OUTPUTS: bit b, equals 1 if successful 


Check if their orders are matched with a corresponding bid order from a client, via Put.MatchOrders. 


Put.MatchOrders 
e INPUTS: 


— the current Storage Market OrderBook 
— query order to match O4 


e OUTPUTS: matching orders 0!..0” 
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Put.ReceivePiece 
e INPUTS: 
— signing key for Mj. 
— current orderbook OrderBook 
— ask order Oasx 
— bid order Opia 
— piece p 
e OUTPUTS: deal order Odea signed by C; and Mj 


3. Seal Storage Miners prepare the pieces for future proofs. 


sector is filled, the sector is sealed. Sealing is a slow, sequential operation that transforms the data 


(the Storage Miner: Sealing is a necessary operation during the Proof-of-Replication as described in 
Section 


Manage.SealSector 
e INPUTS: 


— miner public/private key pair M 
— sector index j 
— allocation table allocTable 


e OUTPUTS: a proof TseaL, a root hash rt 


4. Prove® Storage Miners prove they are storing the committed pieces. 


When Storage Miners are assigned data, they must ®epeatedlyi generate! proofs|of replication to guar- 
antee they restoring the data (for more details, see Section B). Proofs are posted on the blockchain 


and the Network verifies them. 


| Manage.ProveSector 
e INPUTS: 


— miner public/private key pair M 
— sector index j 
— challenge c 


e OUTPUTS: a proof Tpos 


4.3.3 Mining Cycle (for Retrieval Miners) 


We give an informal overview of the mining cycle for Retrieval Miners. 


1. Receive Orders? Retrieval Miners get data requests from the Retrieval Market. 
Retrieval Miners announce their pieces by gossiping their ask orders to the network: they set their- 
price and add an ask order to the market’s orderbook. 


Get.AddOrders 
e INPUTS: list of orders O1..0” 
e OUTPUTS: none 


Then, Retrieval Miners check if their orders are matched with a corresponding bid order from a client. 
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Get.MatchOrders 
e INPUTS: 


— the current Retrieval Market OrderBook 
— query order to match O4 


e OUTPUTS: matching orders 0!..0" 


2. Send: Retrieval Miners send pieces to the client. 


Once\orders\are| matched) Retrieval) Miners|send ithe |piece|toytheyclient (see Section [5.3] for details). 


Put.SendPiece 
e INPUTS: 


— an ask order Oask 
— a bid order Opia 
— a piece p 
e OUTPUTS: a deal order geai signed by M; 


4.3.4 Network Cycle 


We give an informal overview of the operations run by the network. 


1. Assign: The Network assigns clients’ pieces to Storage Miners’ sectors. 
Clients initiate the Put protocol by submitting a bid order in the Storage Market] 
When ask and bid orders match, the involved parties jointly commit to the exchange and submit a 


deal order in the market. At this point, the Network assigns the data to the miner and makes a note 
of it in the allocation table. 


Manage.AssignOrders 
e INPUTS: 


— deal orders Ola- Oa 
— allocation table allocTable 


e OUTPUTS: updated allocation table allocTable’ 


2. Repair: The Network finds faults and attempt to repair them. 
All the storage allocations are public to every participant in the network. At every block, the Network 
checks if the required proofs for each assignment are present, checks that they are valid, and acts 
accordingly: 
e if any proof is missing or invalid, the network penalizes the Storage Miners by taking part of their 
collateral, — 


e cifjaplargeqamountyoftproofsjareimissingyortinvalid (defined by a system parameter ASTE), the 
network considers the Storage Miner faulty, settles the order as failed and reintroduces a new 
“order for the same piece into the the market, 


4Storage orders are submitted via the blockchain, see Section [5] 
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Manage.RepairOrders 
e INPUTS: 
— current time t 
— current ledger £ 
— table of storage allocations allocTable 


e OUTPUTS: orders to repair O},,)-.O%.,, updated allocation table allocTable 


Client Network Miner 
AddOrders(..,Opia) AddOrders(..,Qask) 
MatchOrders(..) 

SendPiece(..,Opbia, p) ReceivePiece(..,Oask) 
AddOrders(Qgeal) AddOrders(..,OQ dea!) 
AddOrder(..,Obia) AddOrder(..,Oask) 
MatchOrders(..) 

ReceivePiece(..,Obia) SendPiece(..,Qask, p) 


ind 


r9 


AddOrders(..,Q deal) AddOrders(..,OQ dea!) 
PledgeSector() 


AssignOrders(..,Qdeal) 


aseuel| 


SealSector‘(..) 
ProveSector(..) 


RepairOrders(..) 


Figure 6: Example execution of the Filecoin DSN, grouped by party and sorted chronologically by row 


4.4 Guarantees and Requirements 


The following are the intuitions on how the Filecoin DSN achieves integrity, retrievability, public verifiability 
and incentive-compatibility. 


e Achieving Integrity) Pieces are named latter their cryptographie hash. After a Put request, Glientsionly 
need to store this hash to retrieve the data via Get and to verify the integrity of the content received. 


e Achieving Retrievability: In a Put request, clients specify the replication factor and the type of erasure | 
Coding desired, specifying in this way the storage to be (f,m)-tolerant. The assumption is that given 
m Storage Miners storing the data, a maximum of f faults are tolerated. By storing data in more than 
one Storage Miner, a client can increase the chances of recovery, in case Storage Miners go offline or 
disappear. 


e Achieving Public Verifiability and Auditability: Storage Miners are required to submit their proofs of 
storage (™seaL, Tpost) to the blockchain. Any user in the network can verify the validity of these 
Proofs) Without having access tothe outsourced data. Since the proofs are stored on the blockchain, 


they are a trace of operation that can be audited at any time. 


e Achieving Incentive Compatibility: Informally, miners are rewarded for the storage they are providing. 
When miners commit to store some data, then they are required to generate proofs. Miners that skip) 


e Achieving Confidentiality: Clients that desire for their data to be stored privately, must encrypt their — 
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Put Protocol Get Protocol 


Market Market 

AddOrders AddOrders 

e INPUTS: list of orders O!..0”7 e INPUTS: list of orders O!..0” 
e OUTPUTS: bit b, equals 1 if successful e OUTPUTS: none 


Set txorder := (O1, .., O”) 1) Gossip O1..0” to the network 


1) 
2) Submit tXorder to L 

3) Wait for tXorder to be included in £ 
4) 


MatchOrders 
Output 1 on success, 0 otherwise © INPUTS: 
— the current Retrieval Market OrderBook 
MatchOrders — query order to match O4 
© INPUTS: e OUTPUTS: matching orders O1..0" 


1) Match each Ot in OrderBook such that: 
a) Check O*.piece is equal to O%.piece 
b) If O4 is an ask order: 

i) Check if O* is bid order 

i) Check O*.price > O4.price 

c) If O4 is a bid order: 

i) Check if O? is ask order 

ii) Check O*.price < O%.price 

2) Output matched orders O1...0" 


— the current Storage Market OrderBook 
— query order to match O4 
e OUTPUTS: matching orders O1..0” 
1) Match each O* in OrderBook such that: 
a) If O4 is an ask order: 
i) Check if O* is bid order 
ii) Check O?.price > O4.price 
iii) Check O* size < O%.space 
b) If O4 is a bid order: 


m. 


i) Check if OŻ is ask order Exchange 
ii) Check O*.price < O%.price SendPi 
iii) Check O* space > O4.size ee 


1 
2) Output matched orders O+...0” uk ask order Osk 


Exchange — a bid order Opia 
SendPiece — apiece p 
e INPUTS: e OUTPUTS: a deal order Oaea signed by Ci 


1) Create Odea: 

a) Set Odgeal-ask := Oask 

b) Set Odea bid = Odeal 

) Get identity of C; from Opig signature 

3) Setup a micropayment channel with C; 

) For each block of data pj of p: 

a) Set mj to be a merkle path from H(p) to pj 
b) Send (Odeal,P5 > Tj) to C; 

c) Receive ( Oadeals j de; 
5) Output Odea 


— an ask order Oask 
— a bid order Opig 
— apiece p 
eè OUTPUTS: a deal order Ogea signed by M; 
1) Get identity of M; from O,,, signature 
2) Send (Oask,Opia,p) to Mi 
3) Receive Ogeai signed by M; 
4) Check if Ogeai is valid according to Definition [5.2} 
5) Output Odeai 


ReceivePiece 
e INPUTS: 
— signing key for Mj. 
— current orderbook OrderBook 
— ask order Oask 
— bid order Opia 
— piece p 
e OUTPUTS: deal order Ogea signed by C; and Mj 
1) Check if Opia is valid: 
a) Check if Opig is in OrderBook 
b) Check if Opig is not referenced by other active Ogeal 
c) Check if Opig-size is equal to |p| 
d) Check if O is signed by M; 
2) Store p locally 
3) Set Odea: = ( Oask; Odeal; H(p) )M; 
4) Get identity of Cj from Obid 
5) Send Odeal to Cj 
6) Output Ogeal 


ReceivePiece 
e INPUTS: 
— aclient’s key Cj 
— an ask order Oask 
— a bid order Opig 
— merkle tree hash of p in the orders hp 
e OUTPUTS: a piece p 
1) Create Odea: 
a) Set Odgeal-ask := Oask 
b) Set Odea -bid = Obid 
) Get identity of M; from Oask signature 
3) Set up a micropayment channel with M; (or re-using 
an existing one) 
4) When receiving (Odea, pj, Tj) from Mi: 
a) Check if Ogeai is valid and matches Osk and Opia 
b) Check if 7; is a valid merkle-path with root hash 
hp 
c) Send ( Odeal, J Yc; 
5) Output p 


Figure 7: Description of the Put and Get Protocols in the Filecoin DSN 
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Manage Protocol 


Network 


AssignOrders 
e INPUTS: 
— deal orders O81 Orval 
— allocation table allocTable 
e OUTPUTS: updated allocation table allocTable’ 
1) Copy allocTable in allocTable’ 
2) For each order Oja: 
a) Check if OF, is valid according to Definition 
b) Get Mj from Oja signature 
c) Add details from ©3,,, to allocTable’ 
3) Output allocTable’ 


RepairOrders 
© INPUTS: 
— current time t 
— current ledger £ 
— table of storage allocations allocTable 
e OUTPUTS: orders to repair OOF, updated alloca- 
tion table allocTable 
1) For each allocEntry in allocTable: 
a) Ift < allocEntry.last + Aproof: skip 
) Update allocEntry.last= t 
) Check if 7 is in Lt—Aproor:t aNd PoSt.Verify (7) 
) On success: update allocEntry.missing= 0 
e) On failure: 
i) update allocEntry.missing++ 
ii) penalize collateral from M,’s pledge 
f) If allocEntry.missing > Afaut then set all the orders 
from the current sector as failed orders 
2) Output failed orders Obeai-Orn and allocTable’. 


deal 


OF 


Oo A 


Miner 
PledgeSector 
e INPUTS: 
— current allocation table allocTable 
— pledge request pledge 
e OUTPUTS: allocTable’ 
1) Copy allocTable to allocTable’ 
2) Set txpledge := (pledge) 
3) Submit txpledge to L 
4) Wait for tXpledge to be included in £ 
5) Add new sector of size pledge.size in allocTable’ 
6) Output allocTable’ 


SealSector 
e INPUTS: 
— miner public/private key pair M 
— sector index j 
— allocation table allocTable 
e OUTPUTS: a proof mseaL, a root hash rt 
1) Find all the pieces p1..pn in sector S; in the allocTable 
2) Set D := pi|p2|-.[pn 
3) Compute (R, rt, nseaL) := PoSt.Setup(M, pkseac, D) 
4) Output TseaL, rt 


ProveSector 
e INPUTS: 
— miner public/private key pair M 
— sector index j 
— challenge c 
e OUTPUTS: a proof Tpos 
1) Find R for sector j 
2) Compute mpost := PoSt.Prove(pkpgst, R, c, Aproot) 
3) Output tpost 


Figure 8: Description of the Manage Protocol in the Filecoin DSN 
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5 Filecoin Storage and Retrieval Markets 


Filecoin has two markets: the Storage Market and the Retrieval Market. The two markets have the same 


structure but different design. Thé!Storage! Market allows! Clients to pay ‘Storage Miners to store data) The 
Retrieval Market allows Clients to retrieve data by paying Retrieval Miners to deliver the data. In both 


cases, clients and miners can set their offer and demand prices or accept current offers. The exchanges are 
tun by the Wetwork= a personification of the network of full nodes in Filecoin. The network guarantees that 
miners are rewarded by the clients when providing the service. 


what is the incentive to run full nodes? 
5.1 Verifiable Markets 


Exchange Markets are protocols that facilitate exchange of a specific good or service. They do this by en- 
abling buyers and sellers to conduct transactions. For our purposes, we require exchanges to be verifiable: 


We present the notion of Verifiable Markets, wher 
Verifiable Market protocols operate the exchange 


of goods/services in a decentralized fashion: ¢6nSistenéy of thelorderbooks) orders settlements and correc) 
execution of services are independently verified via the participants - miners and full nodes in the case of 
“Pilecoii) We simplify verifiable markets to have the following construction: 


Definition 5.1. A verifiable Market is a protocol with (wo phases: order matching and settlement. 


Verifiable Market Protocol 


Figure 9: Generic protocol for Verifiable Markets 


5.2 Storage Market 

The Storage Market is a verifiable market which allows clients (i.e. buyers) to request storage for their data 
and Storage Miners (i.e. sellers) to offer their storage. 

5.2.1 Requirements 


We design the Storage Market protocol accordingly to the following requirements: 


e In-chain orderbook: It is important that: (1) Storage Miners orders are public, so that the lowest. 
_ price is always known to the network and clients can make informed decision on their orders, (2) client 


orders must be always submitted to the orderbook, even when they accept the lowest price, in this way 
ae marker can react tO the mew offer. Hence, we require orders to be dadeam aean torthelPilecoin 
(Dlockchain in order to be added to the orderbook. 
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e Participants committing their resources: We require both parties to commit to their resources 
as a way to avoid disservice: to avoid Storage Miners not providing the service and to avoid clients not 
having available funds. In order to participate to the Storage Market, Storage Miners must pledge, 


depositing a collateral proportional to their amount of storage in DSN (see Section for more 


details). In this way, the Network can penalize Storage Miners that do not provide proofs of storage 
for the pieces they committed to store. Similarly, clients must deposit the finds spécified in the order, 
‘guaranteeing in this way commitment and availability of funds during settlement. 

e Self-organization to handle faults: Orders are only settled if Storage Miners have repeatedly 
proved that they have stored the pieces for the duration of the agreed-upon time period. The Network 


must be able to verify the existence and the correctness of these proofs and act according to the rules 
outlined in the Repair portion of Subsection 


5.2.2 Datastructures 


Put Orders. There are three types of orders: bid Orders, ask orders and deal orders. Storage Miners create 
ask orders to add storage, clients create bid orders to request storage, when both parties agree on a price, 
they jointly create a deal order. The data structures of the orders are shown in detail in Figure[10] and the 


parameters of the orders are explicitly defined. 


Put Orderbook. The Orderbook in the Storage Market is the set of currently valid and open ask, bid and) 
eal orders) Users can interact with the orderbook via the methods defined in the Put protocol: AddOrders, 


MatchOrders as described in Figure[7 


The orderbook is public and everyghonestiuserhasithelsamelviewortheorderbook) Atleveryepoch) new 
orders are added to the orderbook if new order transactions (tXorder) appear in new blockchain blocks; orders 
caro removed if they are cancelled, expired Gr settled? Orders are added in blockchain blocks, hence in the 


orderbook, if they are valid: 
Definition 5.2. We define the validity of bid, ask, deal orders: 
(Walid bid order): A bid order from client Ci, Opia:= (size, funds], price, time, coll, coding])c, is valid if: 


((Walid ask order)? An ask order from Storage Miner M;, Oasx:= (space, price) m, is valid if: 


e M; has pledged to be a miner and the pledge will not expire before time epochs. 
e 
in ask and deal orders). 


@Validqdealyorder): A deal order Ogeai:= (ask, bid, ts)e,,|m; is valid if 


Remark. Ifa malicious client receives a signed deal from a Storage Miner, but never adds it to the orderbook, 
then the Storage Miner cannot re-use the storage committed in the deal. The field ts prevents this attack 
because, after ts, the order becomes invalid and cannot be submitted in the orderbook. 


5This will be a parameter of the system. 
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Storage Market Orders 


bid order 

Opia:= (size, funds|, price, time, coll, coding])c, 

e size, the size of the piece to be stored 

e funds, the total amount that client C; is deposit- 
ing 


e time, the maximum epoch time for which the file 
should be stored] 


e price, the spacetime price in Filecoir{’] 
e coll, the collateral specific to this piece that the 
miner is required to deposit 


e coding, the erasure coding scheme for this piece 


ask order 

Oask: ( space, price ) m; 

e space, amount of space Storage Miner M; is pro- 
viding in the order 


e price, the spacetime price in Filecoin 


deal order 

Oaea: ( ask, bid, ts, hash }c;,m; 

e ask, a cryptographic reference to Oz, from C; 

e order, a cryptographic reference to Opia from M; 

e ts, timestamp epoch in which the order has been 
signed by Mi 

e hash cryptographic hash of the piece that Mj 
will store 


“If not specified, the piece will be stored until expira- 
tion of funds. 

‘Tf not specified, when a Storage Miner is faulty, the 
network can re-introduce the order at the current best 
price. 


Retrieval Market Orders 
bid order 
Opia: ( piece, price )c, 
e piece, the index of the piece requested" 


e price, the price at which C; is paying for one 
retrieval 


ask order 
Oask: ( piece, price ), 
e piece, the index of the piece requested 
e price, the price at which M, is serving the 
piece for 


deal order 
Oaar: { ask, order Ye; m; 


e ask, a cryptographic reference to Oask from 
Ci 

e order, a cryptographic reference to Oa, 
from C; 


"Only pieces stored in Filecoin can be requested 


Figure 10: Orders data structures for the Retrieval and Storage Markets 
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5.2.3 The Storage Market Protocol 


In brief, the" Storage Market protocols divided imltwOlphases order matching and settlement: 


oqOrdersMatching: Clients and Storage Miners stibiiiiti@ieiniord &iSitoleielord ebook SA 
transaction to the blockchain (step 1). When orders are matched, the client sends the piece to the 


— 
n 
et 
© 

se) 
N 

wa 


The Storage Market protocol is explained in detail in Figure 


5.3 Retrieval Market 
The Retrieval Market allows clients to request retrieval of a specific piece and Retrieval Miners to serve it. 


Unlike Storage Miners, Retrieval Miners are not required to store pieces through time or generate proofs of 


storage. Any isein the network can become a Retrieval Miner by Serving pieces in exchange for Filecoin 
tokens. Retrieval Miners can obtain pieces by receiving them directly from clients, by acquiring them from 


the Retrieval Market, or by storing them from being a Storage Miner. 
5.3.1 Requirements 


We design the Retrieval Market protocol accordingly to the following requirements: 


e Off-chain orderbook: Clients must be able to find Retrieval Miners that are serving the required 
Pieces and Mirectly exchange the pieces) Miter Settling on the pricing) This means that the orderbook 
cannot be run via the blockchain - since this would be the bottleneck for fast retrieval requests - instead 
participant will have only partial view of the OrderBook. Hence,we require both parties to gossip their 
orders. | 


e Retrieval withoutitrustéd parties? The impossibility results on fair exchange [IO] remind us that 
it is impossible for two parties to perform an exchange without trusted parties. In the Storage Market, 


the blockchain network acts as a (decentralized) trusted party that verifies the storage provided by 
the Storage Miners) In the Retrieval Market, Retrieval Miners and clients exchange data without the 
network witnessing the exchange of file. We go around this result by equiring the Retrieval’ Minerto 
split their data in multiple parts and for each part sent to the client, they receive a payment. In this 
way, if the client stops paying, or the miner stops sending data, either party can halt the exchange. 
Note that for this to work, WeliituStassuiel thablthereuslalways one honest Retrieval Miner. 


e ‘Payments Channels:) Clients are interested in retrieving the pieces as soon as they submit their 
payments, Retrieval Miners are interested in only serving the pieces if they are sure of receiving a 
payment. Validating payments via a public ledger can be the bottleneck of a retrieval request, hence 


we must rely on @fficient of chain payments) The Filecoin blockchain must support payment channels 
which enableg@apidyoptimistic transactions and use the blockchain only im case of disputes: In this way, 


Retrieval Miners and Clients can quickly send the small payments required by our protocol. Future 
work includes the creation of a network of payment channels as previously seen in [1] [12]. 


5.3.2 Data Structures 


Get Orders) There are three types of orders in the Retrieval Market: clients create bid orders Øpid, Retrieval 
Miners create dskJOrdersiOg,,, and dealyordersyOgegi, are created jointly when a Storage Miner and a client 
agree on a deal. The datastructures of the orders is shown in detail on Figure[10 


Get Orderbook® The Orderbook in the Retrieval Market is he Set or valid’ and open ask bid and deal 
orders. Unlike the Storage Market, every user has a different view of the orderbook, since the orders are 
-gossiped in the network and each miner and client only keep track of the orders they are interested in. 
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Storage Market Protocol 


Order Matching 


1. Storage Miner M; and Client C; add orders to the OrderBook: 
(a) M;i creates Ose Oask”, -- and C; creates Obid! , Obid”, --- 
(b) Orders are submitted to the blockchain via Put.addOrders(O', O?, ..) 
(c) On success, the orders are added to the OrderBook, the funds from C; are deposited and 
the space from M; is reserved. 
2. When orders match, involved parties jointly create Ogea and add it to the OrderBook: 
(a) M; and Cj independently query the OrderBook via Put.matchOrders(Q). 
(b) If M; and C; have matching orders : 
e Cj sends the piece p to M; via Put.SendPiece(Opia, Oask, p) 
e M; receives the piece p from C; via Put.ReceivePiece(Opia, Oask, p)- 
e M; signs Ogeal and sends it to Cj 
(c) Cj signs Ogea and adds it to the OrderBook via Put.addOrders( Odea) 


Settlement 


3. The Network checks if the Storage Miners are correctly storing the pieces: 
(a) When a Storage Miner fills a sector, they seal it (they create a unique replica) via 
Manage.SealSector and submit the proof sea and rt to the blockchain. 


(b) Storage Miners generate new proofs at every epoch and add them to the Filecoin 
blockchain every Aproof epochs via Manage.ProveSectors. 


(c) The Network runs Manage.RepairOrders at every epoch. If proofs are missing or invalid, 
the network tries to repair in the following ways: 


e if any proofs are missing or invalid, it penalizes the Storage Miners by taking part 
of their collateral, 


e if a large amount of proofs are missing or invalid for more than Afay epochs, it 
considers the Storage Miner faulty, settles the order as failed and reintroduces a new 
order for the same piece into the the market, 


e if every Storage Miner storing this piece is faulty, then the piece is lost and the client 
gets refunded. 


4. When the time of the order is expired or funds run out, if the service was correctly provided, 
the Network processes the payments, and removes the orders. 


Figure 11: Detailed Storage Market protocol 
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5.3.3 The Retrieval Market Protocol 


In brief, the Retrieval Market protocol is divided in two phases: order matching and settlement: 


sends to the miner a signed receipt (step 3). The Retrieval Miner presents the delivery receipts to the 


The protocol is explained in details in Figure 


Retrieval Market Protocol 
Order Matching: 


1. Retrieval Miners and Clients add orders to the Get.OrderBook: 
(a) Retrieval Miners M; creates ask orders (Oask', Ok, ..) and Client C; creates bid orders 
(Opia*, Obia”, --)- 
(b) Both M; and C; gossip their orders in the Filecoin network via Get.addOrders 


(c) Since there is no commonly shared orderbook, when users receive orders, they add them 
to their own orderbook’s view. Differently from the Storage Market, these orders are not 
binding and no resource is committed (e.g. clients don’t do any deposit). 


2. When orders match, involved parties jointly create Ogea and add it to the Get.OrderBook: 


(a) Retrieval Miner M; and Client C; independently run Get.matchOrders that queries their 
own current Get.OrderBook view. 


(b) Both M; and C; sign Ogeai and add it to their Get.OrderBook via Get.addOrders (as 
described before) 


(c) C; and M, setup a micropayment channel for Ogeal 
Settlement: 


3. Both parties check whether the piece has been delivered: 


(a) M; sends the piece p in parts via Get.SendPiece 
b) C; receives the p in parts and for each part, C; acknowledges delivery by sending a 
j j 
micropayment via Get.ReceivePiece 


4. When the p has been received by C;, M; can present the micropayments to the network and 
retrieve the payment, both parties remove their orders from the orderbooks. 


Figure 12: Detailed Retrieval Market protocol 
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6 Useful Work Consensus 


The Filecoin DSN protocol can be implemented on top of any consensus protocol that allows for verification 
of the Filecoin’s proofs. In this section, we present how we can bootstrap a consensus protocol based on 


useful work. Instead of wasteful Proof-of- Work computation, the work Filecoin miners do generating Proof 
of-Spacetime is what allows them to participate in the consensus. 


Useful Work. We consider the work done by the miners in a consensus protocol to be Useful if the outcome 
` of the computation is valuable to the network, beyond securing the blockchain. 


6.1 Motivation 


While securing the blockchain is of fundamental importance, Proof-of-Work schemes often require solving 
puzzles whose solutions are not reusable or require a substantial amount wasteful computation to find. 


e Non-reusable Work: Most permissionless blockchains require miners to solve a hard computational 
puzzle, such as inverting a hash function. Often the solutions to these puzzles are useless and do not 
have any inherent value beyond securing the network. Can we re-purpose this work for something 
useful? 


Attempts to re-use work: There have been several attempts to re-use mining power for useful compu- 
tation. Some efforts require miners to perform a special computation alongside the standard Proof- 
of-Work. Other efforts replace Proof-of-Work with useful problems that are still hard to solve. For 
example, Primecoin re-uses miners’ computational power to find new prime numbers, Ethereum re- 
quires miners to execute small programs alongside with Proof-of-Work, and Permacoin offers archival 
services by requiring miners to invert a hash function while proving that some data is being archived. 
Although most of these attempts do perform useful work, the amount of wasteful work is still a preva- 
lent factor in these computations. 


e Wasteful Work: Solving hard puzzles can be really expensive in terms of cost of machinery and 

energy consumed, especially if these puzzles solely rely on computational power. When the mining 
algorithm is embarrassingly parallel, then the prevalent factor to solve the puzzle is computational 
power. Can we reduce the amount of wasteful work? 
Attempts to reduce waste: Ideally, the majority of a network’s resources should be spent on useful work. 
Some efforts require miners to use more energy-efficient solutions. For example, Spacemint requires 
miners to dedicate disk space rather than computation; while more energy efficient, theses disks are 
still “wasted”, since they are filled with random data. Other efforts replace hard to solve puzzles with 
a traditional byzantine agreement based on Proof-of-Stake, where stakeholders vote on the next block 
proportional to their share of currency in the system. 


We set out to design a consensus protocol with a useful work based on storing users’ data. 


6.2 Filecoin Consensus 


We propose a useful work consensus protocol, where the probability that the network elects a miner to create 
@hew block (we refer to this as the Gdfimglpower of the miner) is\proportionalitoltherSstoragelcurrently im 
asein relation tothe rest ofthe nctwork We design the Filecoin protocol such that miners would rather 


invest in storage than in computing power to parallelize the mining computation. Miners offer storage and 


6.2.1 Modeling Mining Power 
Power Fault Tolerance. In our technical report [13], we present Power Faul Tolerant, an abstraction 


that re-frames byzantine faults in terms of participants’ influence over the outcome of the protocol. Every 
participant controls some power of which n is the total power in the network, and f is the fraction of power 
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Power in Filecoin. In Filecoin,the power p. of miner M7 at time vis the sim Of the 17s storage assigns 


In Filecoin, power has the following properties: 


e Public: The total amount of storage currently in use in the network is public. By reading the blockchain, 
anyone can calculate the storage assignments of each miner - hence anyone can calculate the power of 
each miner and the total amount of power at any point in time. 


proving that the service is being provided. By reading the blockehaim, anyone can verify if the power 
claimed by a miner is correct. 


e Variable: At any point in time, miners|eanladd mew storagelim thelmetwork by pledging withiamnew 
sector and filling the sector In this way, miners can change their amount of power they have through 
time. 


6.2.2 Accounting for Power with Proof-of-Spacetime 


_—— miners are required to submit Proofs-of-Spacetime to the network, which are only 
At every 


The power of a miner M; can be calculated and verified by summing the entries in the AllocTable, which 
can be done in two ways: 


e Full§NodelVerification: If a node has the full blockchain log, Gimthe@lNetworkProtocol from the 
genesis block to the current block and read the AllocTable for miner M;. This process verifies every 
Proof-of-Spacetime for the storage currently assigned to M;. 


e Simple Storage Verification: Assume a light client has access to a trusted source that broadcasts — 
the latest block. A light client can request from nodes in the network: (1) the current AllocTable entry 
for miner M;, (2) a Merkle path that proves that the entry was included in the state tree of the last 


LAA 


The seeuritiof the power calculation\comes|fromytheysecurityyofwProoj-of-Spacetimes In this setting, PoSt 
guarantees that ie miner cannot lie about the amount of assigned storage they have. Indeed, 

claim to store more than the data they are storing, since this would require spending time fetching and- 
running the slow PoSt.Setup, and they cannot generate proofs faster by parallelizing the computation, since — 


6.2.3. Using Power to Achieve Consensus 


We foresee multiple strategies for implementing the Filecoin consensus by extending existing (and future) 
PHOGROMSTAKGICONSCHSUS protocols WCC StAKG ISHCPlACeU WIMEASSIGNEMSEOEAEE. While we foresee improve- 


ments in Proof-of-Stake protocols, we propose a construction based on our previous work called 
q@onseisu® [I4]. Our strategy is to 
Expected Consensus. The basic intuition of Expected Consensus EC is to Wererministicallyyy Wnpredictably, 
and secretly elect a small set of leaders at each epoch. On expectation, the number of elected leaders per 
SA proot is a system parameter. 
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ropagating ito thenetworm At each epoch, . In case of a- 
. Although the blocks in chain can be linearly ordered, 

its data structure is a direct acyclic graph. EC is a probabilistic consensus, where each epoch introduces 

history is sufficiently small. 


_ chain where the block belongs to, by extending the chain or by signing blocks. 


Electing Miners. J 
previous protocols: CoA [15], Snow White [I6], and Algorand [17]. 


Definition 6.1. (EC Election in Filecoin) 


this is done similarly to 


In Figure[13} we describe the protocol between a miner (ProveElect) and a network node (VerifyElect). 


This election scheme provides three properties: fairness, secrecy and public verifiability. 
e Fairness: each participant has only one trial for each election, since signatures are deterministic and t 
‘and rand(¢) are fixed! Assuming H is a secure cryptographic hash function, then H((¢||rand(t)) m; )/2” 


must be a real number uniformly chosen from (0, 1). Hence, the probability for the equation to be 
true must be pi / Lyi, which is equal to the miner’s portion of power within the network. Because this) 


probability is linear in power, this likelihood is preserved under splitting or pooling power. Note that 


the random value“fand(#)lis mot! known before time P 


© Secret: 
fiegligible probability, given the assumptions of digital signatures. 


e Public Verifiability: an elected leader i € L' can convince a efficient verifier by showing t, rand(t), 
Al(e\\rand(@))y2"; given the previous point, no efficient adversary can generate a proof without 
having a winning secret key. 


EC Election 


Storage Miner at epoch t Network node on receiving a block at epoch t 


ProveElect(r, t, Mi) > {L, 7$} VerifyElect(mj,t, Mi) + {L|T} 


1. Compute H((t\|r):)/22 < aa 1. Check if 7? is a valid signature from user M; on 
ea t and r 
e on success, output m; = (t, r}i 


2. Check if pt is the power from M; at time t 
A ODE S 3. Test if M; is elected leader H(x$)/2” < Sir 
e on success, output T . 
e otherwise output L 


Figure 13: Leader Election in the Expected Consensus protocol 
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7 Smart Contracts 


Filecoin provides two basic primitives to the end users: Get and Put. These primitives allow clients to store 
data and retrieve data from the markets at their preferred price. While the primitives cover the default use 


cases for Filecoin, we enable for more complex operations to be designed on top of Get and Put by support- 
a emamnranine Users can program new fine-grained storage/retrieval requests that 
we classify as File Contracts as well as generic Smart Contracts. We integrate a Contracts system (based 
on [I8]) and a Bridge system to bring Filecoin storage in other blockchain, and viceversa, to bring other 
blockchains’ functionalities in Filecoin. 


We expect a plethora of smart contracts to exist in the Filecoin ecosystem and we look forward to a community 
of smart-contract developers. 


7.1 Contracts in Filecoin 


Smart Contracts enable users of Filecoin to write|statefiliprograms) that) can spend)tokens)imequest!stor 
age/retrieval of data in the markets and validate storage proofs. Users can i 


nteract with the smart contracts 


Filecoin supports contracts specific to data storage, as well as more generic smart contracts: 
e File Contracts: We allow users to program the conditions for which they are offering or providing 
Storage services) There are several examples worth mentioning: m i 
strategies: clients can design different reward strategies for the miners, for example a contract can pay 


e Smart Contracts: Users can ass0Giatélprogramstoltheimtransactions like in other systems (as in 
Ethereum [18]) whichidomotidirectlywdependjonsthewusejofistoragey We foresee applications such as: 
decentralized naming systems, asset tracking and crowdsale platforms. 


7.2 Integration with other systems 


CBridge are tools that aim ateonnectingMifferentiblockchains; while still workdmpprogressywe plan to support 
(Goss chain interaction in order to bring the Filecoin storage in other blockchain-based platforms as well as 
bringing functionalities from other platforms into Filecoin. 


e Filecoin in other platforms: Other blockchain systems such as Bitcoin [19], Zcash [20] and in 
particular Ethereum and Tezos, allow developers to write smart contracts; however, these platforms 
provide very little storage capability and at a very high cost. We plan to provide a bridge to bring 
storage and retrieval support to these platforms. We note that IPFS is already in use by several 
smart contracts (and protocol tokens) as a way to reference and distribute content. Adding support 
to Filecoin would allow these systems to 


e Other platforms in Filecoin: We plan to provide bridges to connect other blockchain services with 
Filecoin. For example, integration with Zcash would allow support for sending requests for storing 


8 Future Work 


This work presents a clear and cohesive path toward the construction of the Filecoin network; however, we 
also consider this work to be a starting point for future research on decentralized storage systems. In this 
section we identify and populate three categories of future work. This includes work that has been completed 
and merely awaits description and publication, open questions for improving the current protocols, and 
formalization of the protocol. 


8.1 On-going Work 


The following topics represent ongoing work. 
e A specification of the Filecoin state tree in every block. 
e Detailed performance estimates and benchmarks for Filecoin and its components. 
e A full implementable Filecoin protocol specification. 


e A sponsored-retrieval ticketing model where any client C1 can sponsor the download of another client 
C2 by issuing per-piece bearer-spendable tokens. 


e Incremental blockchain snapshotting using SNARK/STARK 

e Filecoin-in-Ethereum interface contracts and protocols. 

e Blockchain archives and inter-blockchain stamping with Braid. 

e Only post Proofs-of-Spacetime on the blockchain for conflict resolution. 


e Formally prove the realizations of the Filecoin DSN and the novel Proofs-of-Storage. 


8.2 Open Questions 
There are a number of open questions whose answers have the potential to substantially improve the network 


as a whole, despite the fact thatqronejofythemshavertoybersolvedsbeforeslauncl» 


e A better primitive for the Proof-of-Replication Seal function, which ideally is O(n) on decode (not 
O(nm)) and publicly-verifiable without requiring SNARK/STARK. 


e A better primitive for the Proof-of-Replication Prove function, which is publicly-verifiable and trans- 
parent without SNARK/STARK. 


e A transparent, publicly-verifiable Proof-of-Retrievability or other Proof-of-Storage. 


e New strategies for retrieval in the Retrieval Market (e.g. based on probabilistic payments, zero knowl- 
edge contingent payments) 


e A better trusted setup scheme for SNARKs that allows incremental expansion of public parameters 
(schemes where a sequence of MPCs can be run, where each additional MPC strictly lowers probability 
of faults and where the output of each MPC is usable for a system). 
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8.3 Proofs and Formal Verification 


Because of the clear value of proofs and formal verification, we plan to prove many properties of the Filecoin 
network and develop formally verified protocol specifications in the coming months and years. A few proofs 
are in progress and more in mind. But it will be hard, long-term work to prove many properties of Filecoin 
(such as scaling, offline). 


e Formulate the Filecoin DSN in the universal composability framework, describing Get, Put and Manage 
as ideal functionalities and prove our realizations. 


Formal model and proofs for automatic SelM Healing guarantees) 


Formally verify protocol descriptions (e.g. TLA+ or Verdi). 


e Formally verify implementations (e.g. Verdi). 
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